Fix endianness issues in OOX crypto routines
authorStephan Bergmann <sbergman@redhat.com>
Thu, 24 Sep 2020 12:51:16 +0000 (14:51 +0200)
committerRene Engelhard <rene@debian.org>
Sun, 18 Oct 2020 08:33:19 +0000 (09:33 +0100)
commitb7864dc193e006cd3a1089c4d65487f4c6f98ff0
tree38dd57b2fd63bf9e81fdb26551e7a2a2cde34e00
parent900c31cfb2efe7fd57a9f73e1bc18f19b3133424
Fix endianness issues in OOX crypto routines

...without which CppunitTest_sw_ooxmlencryption failed on (big-endian) s390x:

* The 32-bit segment counter in AgileEngine::de-/encrypt apparently needs to be
  stored in LSB format (at least, if it is, CppunitTest_sw_ooxmlencryption
  ultimately succeeded, whereas otherwise it failed).

* The UTF-16 string in Standard2007Engine::calculateEncryptionKey apparently
  needs to be in LSB format (at least, if it is, CppunitTest_sw_ooxmlencryption
  ultimately succeeded, whereas otherwise it failed).

* The various 32-bit values in the EncryptionStandardHeader and
  EncryptionVerifierAES data structures apparently need to be written out in LSB
  format in Standard2007Engine::writeEncryptionInfo, given that they are always
  read in LSB format in Standard2007Engine::readEncryptionInfo.

Change-Id: I3a1efbfe324b1bbd539b88dc5d40bb44f9676ffa
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/103315
Tested-by: Jenkins
Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
Gbp-Pq: Name oox-crypto-bigendian.diff
oox/source/crypto/AgileEngine.cxx
oox/source/crypto/Standard2007Engine.cxx